Research and applications: 



The effectiveness of a new generation of 
computerized drug alerts in reducing the risk of injury 
from drug side effects: a cluster randomized trial 



Robyn TamblynJ'^ Tewodros Eguale,' David L Buckeridge,''^ Allen Huang, 



1,2 



James HanleyJ'^ Kristen Reidel,' Sherry Shi,' Nancy Winslade' 



^Department of Epidemiology Et 
Biostatistics, McGill University, 
Montreal, Quebec, Canada 
^Department of Medicine, 
McGill University, Montreal, 
Quebec, Canada 



Correspondence to 

Dr Robyn Tamblyn, McGill 
University, Morrice House, 1140 
Pine Ave W, Montreal, Quebec 
H3A 1A3, Canada; 
robyn.tamblyn(a)mcgill.ca 



Received 23 September 201 1 
Accepted 16 December 2011 
Published Online First 
12 January 2012 



ABSTRACT 

Context Computerized drug alerts for psychotropic drugs 
are expected to reduce fall-related injuries in older 
adults. However, physicians over-ride most alerts 
because they believe the benefit of the drugs exceeds 
the risk. 

Objective To determine whether computerized 
prescribing decision support with patient-specific risk 
estimates would increase physician response to 
psychotropic drug alerts and reduce injury risk in older 
people. 

Design Cluster randomized controlled trial of 81 family 
physicians and 5628 of their patients aged 65 and older 
who were prescribed psychotropic medication. 
Intervention Intervention physicians received 
information about patient-specific risk of injury computed 
at the time of each visit using statistical models of non- 
modifiable risk factors and psychotropic drug doses. Risk 
thermometers presented changes in absolute and 
relative risk with each change in drug treatment. Control 
physicians received commercial drug alerts. 
Main outcome measures Injury risk at the end of 
follow-up based on psychotropic drug doses and non- 
modifiable risk factors. Electronic health records and 
provincial insurance administrative data were used to 
measure outcomes. 

Results Mean patient age was 75.2 years. Baseline risk 
of injury was 3.94 per 100 patients per year. 
Intermediate-acting benzodiazepines (56.2%) were the 
most common psychotropic drug. Intervention physicians 
reviewed therapy in 83.3% of visits and modified therapy 
in 24.6%. The intervention reduced the risk of injury by 
1.7 injuries per 1000 patients (95% CI 0.2/1000 to 3.2/ 
1000; p=0.02). The effect of the intervention was 
greater for patients with higher baseline risks of injury 
(p<0.03). 

Conclusion Patient-specific risk estimates provide an 
effective method of reducing the risk of injury for high- 
risk older people. 

Trial registration number clinicaltrials.gov Identifier: 
NCT00818285. 
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INTRODUCTION 

Injuries are among the leading causes of morbidity 
and mortality in older adults.' ^ The majority of 
injuries are fail-related, and 5—10% are fatal. A 
further 9—27% lead to a permanent loss of capacity 
for independent living.^ ^ Psychotropic drugs are 
a potentially preventable cause of injury. ^^'^ These 
drugs are commonly used in older adults, often for 
indications, such as insomnia and pain, where 
strong evidence of efficacy is lacking. '^^'® Based on 



systematic reviews, the risk of injury is increased 
by 39%, 59%, and 50% with the use of benzodi- 
azepines, antidepressants, and antipsychotics, 
respectively.'' Risks appear to be dose-dependent, 
particularly for antipsychotics and opioids, where 
the most rapid increase in use is seen for older 
adults.7 

Effective management of psychotropic medica- 
tion is challenging. In older adults, 21—33% of 
prescribed psychotropic medication is relatively 
contraindicated,^"^^^ and 29% in doses that exceed 
those recommended.^^ Moreover, 20% of older 
adults use more than one psychotropic drug 
concurrently, and 69% have more than one physi- 
cian prescribing treatment, increasing the risk of 
undetected cumulative toxicity.^^ 

Computerized prescribing and decision support 
are expected to address preventable medication 
errors, as these digital technologies can guide dosing 
and provide alerts on drug treatment duplication, 
contraindications, and drug interaction errors, espe- 
cially when integrated with information on all 
dispensed medication. However, the majority of 
drug alerts are over-ridden, particularly for psycho- 
tropic medication^''^^^ in both hospital-based and 
community-based studies. Even when drug alert 
systems are customized to present only clinically 
important interactions, physicians over-ride the 
majority of alerts, because they are deemed not 
clinically relevant and/or the benefit is believed to 
exceed the risk.^"*^^* Yet, the patient-specific 

risk is rarely known, even though it can be estimated 
by incorporating into drug alert systems predictive 
models of adverse events developed through phar- 
macoepidemiological studies.^" The advanced 
computing power available in today's electronic 
record systems and the focus on individualized 
medicine provides an unprecedented opportunity 
to integrate detailed patient data into complex 
predictive models for estimating patient risk. 

In this study, we tested the hypothesis that the 
incorporation of patient-specific risk estimates 
into a computerized prescribing decision-support 
system in primary care would increase physician 
response to alerts for psychotropic medication and 
reduce the risk of psychotropic drug-related injury 
in older adults, particularly for patients with 
a higher baseline risk. 

METHODS 
Context 

The study was conducted in Quebec, Canada, 
a province with 8.5 million residents and 16000 
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physicians. The provincial insurance agency (RAMQ) provides 
health insurance for all provincial residents, and pays all physi- 
cians and community pharmacies on a fee-for-service basis. 
Beneficiary, medical billing and pharmacy claims data can be 
used to create longitudinal health histories for each patient. 
These data have been validated and are often used for health 
services and epidemiological research. In 2003, MOXXI, an 
experimental community-based clinical information system, 
was the first to link to these databases and integrate this 
information into electronic health record systems to support 
clinical decision-making.^^ 

Design and study population 

A cluster randomized controlled trial was conducted to test the 
hypothesis that prescribing decision support that provided 
patient-specific risk assessment would reduce the risk of injury in 
older adults. The trial was conducted in a population of 81 
family physicians and 5628 of their older patients from 
September 2008 to July 2010. This sample size was expected to 
detect a difference in risk of 5% assuming 80 physician clusters, 
50 patients per physician, a cluster correlation of 0.01, and a Type 
I and II error of 5% and 20%, respectively. Injury risk reduction 
was assessed at the end of the follow-up period (July 2010). 

Potentially eligible physicians were identified from the roster 
of active primary care physicians in the Quebec provincial 
regulatory authority and invited to participate in the MOXXI 
primary care research network to evaluate the use of comput- 
erized drug and chronic disease management systems in primary 
care.-'''^** ^'^ To be enrolled in the network, physicians 

needed to practice actively in an urban community setting in 
Montreal or Quebec City, and use the computerized drug 
management system successfully to write a minimum of 10 
patient prescriptions per week. 

Patients were eligible if they were age 65 years or older, had an 
active dispensed prescription for a psychotropic drug, or were 
prescribed a new psychotropic drug at a visit during the follow- 
up period. Psychotropic drugs included those with central 
nervous system side effects that increased the risk of injury: 
benzodiazepines, antidepressants, antipsychotics, anticonvul- 
sants, antihistamines, and opiates.^^ 

Randomization and blinding 

To optimize balance in the intervention and control group, we 
stratified physicians by the number of patients in their practice 
who were prescribed psychotropic medication in the past year, 
with a minimum of two physicians per stratum. Within each 
stratum, an equivalent number of physicians were randomized 
by the biostatistician to intervention and control groups 
using a random number table. Physicians were not blinded to 
the intervention status, but were blinded to the specific 
study outcomes that were measured. Unless advised by their 
physicians, patients were blinded to the intervention status. 

Intervention and control group 

Both the intervention and control groups used the MOXXI 
community-based clinical information system (CIS). A link to 
the provincial insurance agency (RAMQ) was used to pre- 
populate demographic information for the practice population 
based on a study physician's billings from the previous year. For 
patients who consented to participate in the research network, 
all medical services and prescription drugs provided for the past 
year were loaded into the MOXXI CIS, and thereafter all new 
records of medical services and prescriptions were refreshed on 
a daily basis. These data were used to update the profile of 



dispensed medications, dates and reasons for emergency depart- 
ment visits and hospitalizations, medical and surgical proce- 
dures, and health problems.^' A commercial drug alert system 
(http://wvvw.vigilance.ca) automatically reviewed each new 
prescription for potential contraindications, including therapy 
duplication, dosing error, cumulative toxicity, and drug— disease, 
drug— drug, and drug— allergy interactions. Physicians could set 
the threshold for the alert system to one of three levels (1, severe 
alerts only; 2, moderate and severe alerts; 8, all alerts), which 
restricted alerts generated automatically during the prescribing 
process. By default, the system was set to level 2. However, all 
alerts generated for a patient were available for the physician to 
review in a drug alert summary in the patient's electronic chart. 
Physicians randomized to the control and intervention groups 
had access to these standard features of the MOXXI CIS. 

Physicians randomized to the intervention group received 
a patient-specific risk of injury alert when a patient was prescribed 
a psychotropic medication that increased the risk of injury. The 
personalized alert used a published predictive model^^ to estimate 
the risk of injury based on the patient's age, sex, injury history, 
presence of cognitive impairment, gait, and balance problems, and 
doses of selected psychotropic medication (selective serotonin / 
nor-epinephrine reuptake inhibitors antidepressants, antipsy- 
chotics, low-, intermediate- and high-potency opiates, interme- 
diate- and long-acting benzodiazepines, anticonvulsants, and 
first-generation antihistamines"*' ^^). We set a relatively low 
threshold for showing the alert — an increase in risk of 1 per 
1000 — to enable us to assess the likelihood of changes in drug 
treatment as a function of the magnitude of patient risk. 

Craphics, in the form of risk thermometers, were created to 
show physicians the patient's risk of injury in the next 
12 months related to psychotropic medication as well as non- 
modifiable characteristics (eg, age, sex) (figure lA). Physicians 
would see the risk thermometer, annotated with numeric values 
generated by the risk calculation described above, when they 
opened the patient drug profile or when they prescribed 
a psychotropic drug. Drugs that contributed to the risk calcu- 
lation were highlighted m the patient's drug profile. If the 
physician attempted to reduce the risk of injury by stopping or 
decreasing the dose of a psychotropic medication, the absolute 
and relative reduction in risk would be shown as an adjustment 
in the level of the thermometer and a change in the numeric 
values (figure IB). If a new psychotropic drug was started or the 
dose was increased, the absolute and relative increase in the risk 
would be shown (figure IC). If no change in medication was 
instituted (or the risk was increased by a medication change), 
physicians had to select a reason for the decision from a stan- 
dardized pick-list (eg, prescribed by another physician). A refer- 
ence section was available with publications on the risk of injury 
related to psychotropic drug use and methods of tapering 
benzodiazepines. 

Physicians in the intervention group received a 5-minute 
training program on the risk of injury alert that outlined how to 
interpret the risk thermometer information, the expected 
changes in risk with new or discontinued medication, the 
completion of reasons for not changing therapy if applicable, 
and the location of reference information. Physicians were 
advised that stopping or reducing the dose of highlighted 
medications in the patient's drug profile would reduce the 
patient's risk of injury. 

Physician characteristics 

Physician age, sex, and years of practice experience were docu- 
mented at enrollment. Annual measures of practice size, number 
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Figure 1 (A) Screenshot of the user interface for the 'risk of injury alert' showing current risk and lowest possible risk. (B) Screenshot of the user 
interface for the risk of injury alert showing a reduction in risk due to modified treatment. (C) Screenshot of the user interface for the risk of injury alert 
showing risk increase with modified treatment. 
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of practice settings, workdays, and daily practice volume 
were calculated for the year before randomization from each 
physician's RAMQ billing data using methods developed 
previously.'"' ''^ Utilization of the drug management component 
of the MOXXI CIS was measured by calculating the number of 
electronic prescriptions written per 100 patient visits billed in 
the year before randomization. Skill in using the prescribing 
system was assessed by timed assessment of prescribing speed 
for four standardized prescriptions 3 months after the initial 
training for the MOXXI CIS. 

Patient characteristics 

Patient age, sex, and residential address were documented at 
enrollment from the RAMQ insurance records. Residential 
address was linked to neighborhood-level Statistics Canada 
census information to obtain estimates of patient household 
income, classified by low- (<|31 753), middle- ($31 754- 
$80000) and high- (>$80000) income neighborhoods. Relevant 
comorbidity was measured using diagnostic and procedure codes 
in RAMQ billing claims and the electronic health record problem 
list for each patient. Injury history was assessed by calculating 
the number of temporally distinct episodes of billing claim 
treatments for fractures and soft-tissue injuries in the past 
24 months using algorithms that were previously validated. 
Cognitive impairment and gait and balance problems were 
defined in accordance with International Classification of Disease 
Ninth Revision (ICD9) codes included in the predictive model 
for injury.^® Cognitive impairment included dementia (291), 
alcohol-related cognitive impairment (292), and Alzheimer's and 
other cerebral degeneration (331). Cait and balance problems 
included degenerative diseases of the extrapyramidal system 
(332 333), seizure disorders (345), vertiginous syndromes (386), 
syncope (780), and orthostatic hypotension (458). The Charlson 
comorbidity index,'*'' ''^ number of emergency department visits, 
and number of hospital admissions in the past year were also 
measured using RAMQ billing data to provide descriptive 
information about population characteristics. 

Outcomes 

The primary study outcome was the risk of injury at the end of 
the follow-up period. To assess whether the effect of the inter- 
vention was greater for patients at higher risk, each patient's risk 
of injury was measured twice, once at the patient's first visit 
after randomization (baseline risk) and again on the last day of 
follow-up (July 15, 2010) for the intervention and control groups 
(outcome risk). Overall injury risk was measured using the 
predictive model used in the intervention, which reflected the 
probability of an injury in the next year based on age, sex, 
comorbidity, and daily dispensed doses of benzodiazepines, 
opiates, antipsychotics, antidepressants, anticonvulsants, and 
antihistamines (figure 2). Drug doses were based on active 
dispensed prescriptions on the first and last day of foUow-up. To 
permit multiple drugs within the same therapeutic category to 
be combined, doses were standardized, using the same approach 
as the predictive model, by dividing the daily dose by the WHO- 

Flgure 2 Formula for estimating the 
risl< of injury based on individual patient 
characteristics.^' NB Baseline risk was 
specified at 2.06%, representing the 
incidence of injury among 65-year-old 
men without any other risk factors. 
SSRI, selective serotonin reuptake 
inhibitors; SSNRI, selective serotonin- 
norepinephrine reuptake inhibitors. 



recommended daily dose for adults for each drug. For example, 
a patient prescribed 20 mg diazepam and 10 mg lorazepam daily 
would be using three standardized doses of benzodiazepine 
(20 mg diazepam/10 mg (WHO-recommended dose) -I- 10 mg 
lorazepam/ 1 0 mg (WHO-recommended dose) = three doses). 

In a secondary analysis, we assessed the physician's response 
to the injury risk alert, and changes in the use and dose of 
psychotropic medications in the intervention and control 
groups. Alert response was retrieved from the MOXXI audit trail 
for physicians in the intervention group. The audit trail recorded 
each action taken in relationship to psychotropic medication 
and its impact on overall and modifiable risk. Actions were 
classified into three mutually exclusive categories: (1) over-rode 
alert, (2) reviewed options but made no change, and (3) modified 
dose or drug to reduce risk. Reasons for over-riding alerts were 
tabulated. 

To assess change in use and dose of medication, we estimated 
differences between the intervention and control groups in (1) the 
number of active psychotropic medications prescribed or 
dispensed, and (2) the standardized dose of psychotropic medi- 
cation by therapeutic category between the baseline and follow- 
up period. Data were retrieved from records of all medications 
dispensed from the daily updates of the drug profile by the RAMQ. 

Analysis 

To test the hypothesis that the patients in the intervention group 
would have a greater reduction in the risk of injury than patients 
in the control group, we used multivariate linear regression 
within a generalized estimating equation framework. An 
exchangeable correlation structure was used to account for clus- 
tering of patients within physician. Each patient's calculated risk 
of injury on the last day of follow-up was the continuous 
outcome, and experimental status (intervention vs control) was 
the predictor. The patient's baseline risk score was used to test the 
hypothesis that the benefits of the intervention would be greater 
for high-risk patients. The interaction term between baseline risk 
score and experimental status was included in the model, and the 
Wald stxatistic was used to assess statistical significance. We 
used the same approach to estimate the change in the number of 
psychotropic medications and dose by therapeutic category 
between the baseline and follow-up period. For medication dose, 
we estimated one model per therapeutic category. 

RESULTS 

Among the 410 physicians eligible, 81 consented to participate, 
representing 5628 eligible patients. All consenting physicians 
and their patients were included in the analysis (figure 3).''^ 
Physicians in the intervention and control groups were 
predominantly male, French speaking, and in practice for 25 or 
more years (table 1). Physicians in both groups worked, on 
average, in 2.2 practice settings, saw 16—17 patients per day, and 
used the MOXXI CIS with equivalent frequency and speed of 
writing electronic prescriptions. 

Patients in the intervention and control groups were similar: 
on average 75— 76 years of age, predominantly female, and 
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Figure 3 Consort diagram of 
physicians and patients eligible and 
enrolled in the trial. 
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residing in middle-income neighborhoods (table 1). Approxi- 
mately 5% of patients had treatment for at least one injury 
in the past year, and a similar proportion had cognitive 
impairment. The prevalence of gait and balance problems was 
slightly higher in the control group as were the proportion of 
patients with at least one emergency department visit in the 
past year 

At the first visit after randomization, the most prevalent 
psychoactive medication used was an intermediate-acting 
benzodiazepine, taken by approximately one-third of patients, 
followed by antidepressants, anticonvulsants, and antipsy- 
chotics (table 2). For intermediate-acting benzodiazepines, the 
mean daily dose prescribed was 10—20% over the recommended 
adult dose, primarily because 4.6% of patients in the control 
group and 3.8% in the intervention group were using more than 
one intermediate-acting benzodiazepine concurrently. This was 
also true for antidepressants, where 1 — 1.2% of patients were 
using more than one antidepressant. The overall risk of injury at 
baseline was comparable in the two groups — the risk of injury in 
the next year was 4.03 per 100 in the control group, and 3.85 in 
the intervention group, of which an equivalent amount (control, 
0.51; intervention, 0.49) was related to the use of psychoactive 



medication. The distribution was skewed, with a median of 
3.55, an IQR of 2.88-4.51, and a range 2.09 to 45.30. 

Among the 2887 injury alerts generated at the first visit for 
patients in the intervention group, 13.7% were for new 
psychoactive medications that were being started by the 
physician during the visit, and the remaining alerts (86.3%) were 
for existing medications (table 3). Physicians responded to 2404 
(83.3%) risk of injury alerts, by reviewing options but deciding 
not to change the treatment (n=1694, 58.7%), by reducing the 
dose, or by discontinuing psychoactive medications (n=710, 
24.6%). Of interest, physicians were more likely to change the 
treatment plan for existing medication (25.7%) than for newly 
started therapy (17.7%). The most common reason for not 
changing treatment was the belief that the benefit would exceed 
the risk (n=1840, 84.5%), followed by patient resistance to 
change in therapy (n=179, 8.2%). While the majority of physi- 
cians changed their alert settings from the default setting to 
view only the most severe alerts (table 1), there were no 
statistically significant differences in the alert levels selected by 
the control and intervention groups. 

On average, patients in the intervention and control groups 
were followed for 467 and 452 days, respectively. Although there 
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Table 1 Characteristics of the 81 physicians and the 5628 patients in 
the intervention and control groups 



Intervention, 
Control, N = 40 N = 41 



Physician demographics 


N 


% 


N 


% 


Sex 










Male 


20 


50.0 


22 


55.0 


Female 


20 


50.0 


18 


45.0 


Language 










English 


14 


35.0 


12 


30.0 


French 


26 


65.0 


28 


70.0 


Practice experience (years) 










<15 


5 


12.5 


4 


10.0 


15-24 


11 


27.5 


10 


25.0 


£25 


24 


60.0 


26 


65.0 


Practice characteristics 


Mean 


SD 


Mean 


SD 


Annual practice size 


1137.4 


555.9 


1330.6 


711.6 


Number of practice settings 


2.2 


1.3 


2.2 


1.0 


IMumber of clinic days worked 


190.6 


43.6 


186.6 


39.1 


Number of patients/ clinic day 


16.7 


7.6 


17.4 


6.5 


Use of MOXXI CIS 


Mean 


SD 


Mean 


SD 


Electronic prescription/100 visits 


40.5 


28.7 


39.2 


24.4 


Speed in prescription 4 standard script! 


; (min) 3.08 


0.88 


3.04 


1.09 


Minimum alert setting 










Level 1 (severe only) 


31 


77.5 


24 


58.5 


Level 2 (moderate to severe) 


5 


12.5 


12 


29.3 


Level 3 (all alerts) 


4 


10.0 


5 


12.2 








Intervention, 




Control, N = 


2741 


N = 2887 




Patient demographics 


Mean 


SD 


Mean 


SD 


Age (years) 


75.63 


7.47 


74.81 


6.84 


Patient sex 


N 


% 


N 


% 


Female 


1876 


68.4 


1901 


65.8 


Male 


865 


31.6 


986 


34.2 


Socioeconomic status 










Low income (<$31 753) 


868 


31.7 


790 


27.4 


Middle income ($31 754-$80 000) 


1682 


61.3 


1918 


66.4 


High income (>$80 GOO) 


191 


7.0 


179 


6.2 


Patient comorbidity 










Number injuries past 24 months 










No injury 


2618 


95.5 


2766 


95.8 


1 injury 


108 


3.9 


107 


3.7 


2 injuries 


10 


0.4 


13 


0.5 


a3 injuries 


5 


0.2 


1 


0.0 


Gait or balance problem 


505 


18.4 


476 


16.5 


Cognitive impairment 


150 


5.5 


130 


4.5 


Charlson comorbidity index 










0 


1379 


50.3 


1565 


54.2 


1-2 


1039 


37.9 


1048 


36.3 


3-4 


191 


7.0 


166 


5.7 


>5 


132 


4.8 


108 


3.7 


Healthcare utilization 










>1 Emergency department visit 


1011 


36.9 


927 


32.1 


^:^ Hospitalization 


617 


22.5 


608 


21.1 



was an overall trend for the number of drugs and doses in all 
psychotherapeutic classes (except intermediate-acting benzodi- 
azepines) to be reduced in the intervention group, no single class 
decreased significantly in dose, when modeled alone (table 4). 
However, there were significant interactions between baseline 
risl< and intervention status for antipsychotics (interaction term: 
p=0.02) and anticonvulsants (interaction term: p=0.03), 
with reductions in doses occurring for patients with greater 
baseline risk. The most prevalent medication affected in these 



Table 2 Psychoactive medication at the first visit after randomization 



Control, Intervention, 
N = 2741 N=2887 



Distribution by tberapeutic class 


N 


% 


N 


% 


Benzodiazepines 










Intermediate-acting 


1516 


55 


1648 


57 


Long-acting 


109 


4 


111 


4 


Antidepressants 


712 


26 


721 


25 


Anticonvulsants 


451 


16 


404 


14 


Antipsychotics 


258 


9 


278 


10 


Opiates 










High potency 


22 


1 


5 


0 


Intermediate potency 


96 


4 


104 


4 


Low potency 


174 


6 


195 


7 


Antihistamines 


7 


0 


4 


0 


Standardized daily dose among users 


Mean 


SD 


Mean 


SD 


Benzodiazepines 










Intermediate-acting 


1.20 


1.71 


1.10 


1.52 


Long-acting 


0.81 


0.45 


0.74 


0.37 


Antidepressants 


1.04 


0.55 


1.06 


0.58 


Anticonvulsants 


0.51 


0.41 


0.54 


0.42 


Antipsychotics 


0.59 


2.02 


0.79 


3.38 


Opiates 










High potency 


0.29 


0.85 


0.16 


0.15 


Intermediate potency 


0.43 


0.38 


0.50 


0.63 


Low potency 


0.88 


0.77 


0.89 


0.70 


Antihistamines 


0.13 


0.04 


0.79 


3.38 


Overall use and risk 


Mean 


SD 


Mean 


SD 


Number of psychoactive medications 


1.28 


0.61 


1.25 


0.58 


Proportion prescribed by study doctor 


0.81 


0.38 


0.78 


0.41 


Combined standardized dose 


1.18 


1.59 


1.15 


1.69 


Overall risk of injury 


4.03 


1.87 


3.85 


1.70 


Risk related to psychoactive medication 


0.51 


0.80 


0.49 


1.14 



therapeutic groups was risperidone for antipsychotics (37.5% of 
patients (n=201)) and pregabalin and gabapentin for anticon- 
vulsants (71.1% of patients (n=608)). The most common indi- 
cations for prescribing antipsychotics was psychosis/bipolar 
disorder (32.2% of patients) and dementia/delirium (20.4% of 
patients), and for anticonvulsants it was pain (72.6%). 

Congruent with these changes in therapy, the risl< of injury in 
the intervention group was reduced by 1.7 injuries per 1000 
patients (95% CI 0.2/1000 to 3.2/1000; p=0.02) compared with 



Table 3 Response to the TRIPP alert in the intervention group by type 

of prescription 

Renewing an 

existing Starting a new 

psychoactive psychoactive 
medication medication 





N 


% 


N 


% 


Number of alerts 


2491 




396 




Changed prescription 


640 


25.7 


70 


17.7 


Reviewed but no change 


1399 


56.2 


295 


74.5 


l\lo review + over-rode alert 


452 


18.1 


31 


7.8 


Reasons for over-riding alert 










Benefit greater than risk 


1545 


62.02 


295 


74.49 


Patient demand/resistance 


161 


6.46 


18 


4.55 


Need to consult prescribing doctor 


53 


2.13 


3 


0.76 


Will review next time 


47 


1.89 


3 


0.76 


Drug information incorrect 


45 


1.81 


7 


1.77 



640 
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Table 4 Change in intermediate outcomes: psychotropic drug dose and number of drugs 



Intermediate outcomes 


Dose at the end of follow-up[y] 


Dose change[y] 
(follow-up— baseline) 


Cluster adjusted difference[z, x] 


lnteraction[x] 

Udacllllc riaK A 

intervention 




Control 


Intervention 


Control 


Intervention 








Dose by therapeutic class[*] 


Mean±SD 


Mean±SD 


IVIean±SD 


Mean±SD 


Difference 


95% CI 


p Value 


Benzodiazepines 
















Intermediate-acting 


0.68±0.6 


0.66±0.6 


-0.79±1.7 


-0.74±1.5 


-0.008 


-0.05 to 0.03 


0.91 


Long-acting 


0.88±0.5 


0.70±0.3 


-0.32±0.6 


-0.43±0.5 


-0.006 


-0.01- to -0.00 


0.19 


Antidepressants 


1.04±0.5 


1.07±0.6 


-0.41 ±0.8 


-0.46±0.8 


-0.011 


-0.03 to 0.01 


0.48 


Anticonvulsants 


0.51 ±0.4 


0.59±0.4 


-0.21 ±0.5 


-0.19±0.4 


0.006 


-0.00 to 0.01 


0.03 


Antipsychotics 


0.44±0.4 


0.54±0.7 


-0.33±1.9 


-0.49±3.2 


0.005 


-0.00 to 0.01 


0.02 


Opiates 
















Intermediate potency 


0.38±0.4 


0.56±0.6 


-0.24±0.5 


-0.26±0.7 


0.001 


-0.00 to 0.01 


0.65 


Low potency 


0.79±1.2 


0.49±0.5 


-0.63±1.0 


-0.70±0.8 


-0.004 


-0.01 to 0.00 


0.29 




Number drugs at the end of 
follow-up 


Drug change 
(follow-up — baseline) 


Cluster adjusted difference 


Interaction 
baseline risk x 
intervention 


By number of psychotropic 


Control 


Intervention 


Control 


Intervention 








drugs 


Mean±SD 


Mean±SD 


Mean±SD 


Mean±SD 


Difference 


95% CI 


p Value 




0.67±0.8 


0.62±0.7 


-0.60±0.8 


-0.63±0.8 


-0.02 


-0.09 to 0.05 


0.96 



*Th8re was an insufficient number of patients on high-potency opiates (n=27) or antihistamines (n=ll) to mode! separately. 
tStandardized drug dosage = daily prescribed dosage (mg)/WHO-recommended adult dosage (mg). 

lEach therapy class was modeled separately for drug dose in a model that included baseline dose and intervention group status. To test the hypothesis that the intervention was modified by 
baseline risk, baseline risk was added to the model as well as a two-way interaction term between intervention group status and baseline risk. 

§Change in the number of psychotropic drugs was assessed in a model that included baseline number of psychotropic drugs and Intervention group status. To test the hypothesis that the 
intervention was modified by baseline risk, baseline risk was added to the model as well as a two-way interaction term between intervention group status and baseline risk. 

the control group (figure 4). The effect of the intervention was injury and the intervention: p=0.03) (figure 4). For patients in 
significantly greater for patients at higher risk of injury at the upper decile of risk of 5.76, the intervention produced 
the start of the trial (interaction between the baseline risk of a reduction in risk of 5.3 per 1000 (95% CI 3.8 to 6.8). 



Control Intervention Cluster adjusted reduction in the 
N=2741 N=2887 risk of injury per 1000 





Mean 


SD 


Mean 


SD 


Reduction 


95% CI 
(p Value) 


Number of days of follow-up 


467 


178 


452 


181 








3.77 


1.73 


3.58 


1.31 


-0.17 


-0.32. -0.02 


Risk of injur>' at the end of follow-up 












(0.02) 



9 ■ 



1 ■ 




3.71 is the point where the effect of the 
risk of injury alert is the same as the 
conventional alert 



4.5 5 5.5 6 6.5 7 7.5 8 
Risk at baseline (K probability of injury in the next year) 

t Intervention —•—Control 



Figure 4 Risl< of injury at ttie enti of follow-up in the intervention and control groups and modification of the effect of the intervention by the 
magnitude of the baseline risk. The model used to estimate the change in the effect of the intervention by baseline risk of injury was: follow-up risk 
(y)=intercept (0.3696) -I- baseline risk (0.8427) -I- intervention (0.9585) -I- baseline risk X intervention (-0.2584) (p=0). 
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DISCUSSION 

This study tested the benefits of using a new approach for drug 
safety alerts that provided patient-specific risk estimates of the 
likelihood of harm. The provision of patient-specific risk infor- 
mation resulted in a therapy review in over 80% of patients and 
significant reductions in the risk of injury related to psychoac- 
tive medication, particularly for those who were at greatest risk 
of fall-related injuries. 

This study produced a greater response to alerts than previous 
studies,^'*"^'' even those that used highly selected subsets of 
clinically relevant alerts.''^ Not only did physicians review 
consequences of revising treatment for the vast majority of 
patients, they also modified drug therapy in one-quarter of 
instances. 

Of interest, the most common reason for not changing 
therapy was that physicians perceived the benefit of treatment 
to be greater than the risk. This reason was particularly common 
when the patient was starting new medication, possibly because 
a physician who starts a patient on medication will generally 
have decided that the benefit exceeds the risk before prescribing, 
even if the precise risk and benefit are not known. In contrast, 
prescription renewals for prevalent users were more likely to be 
modified, possibly because the effects of treatment (or lack 
thereof) can be more precisely evaluated, and thus more precise 
risk information can be factored into renewal decisions. It may 
also be because the evidence supporting the benefits of treat- 
ment is less compelling to start with, particularly in some 
therapeutic categories. The most significant reductions in 
therapy in this study were for antipsychotics and anticonvul- 
sants, therapeutic categories that have been reported to have 
both high levels of off-label use and weak to non-existent 
evidence of benefit.^" Indeed, 22% of anticonvulsant prescrip- 
tions were for gabapentin, and all of these prescriptions were for 
off-label use for pain. 

A ubiquitous complaint about drug alerts is that there are too 
many 'nuisance alerts', ie, alerts for problems of little clinical 
significance.^^ This study is the first to identify a data- 
driven approach to identifying clinically meaningful alerts and 
quantifying the risk associated with an alert. Typically, 'expert 
opinion' is used to classify alerts as mild, moderate, and severe.'*^ 
This method of classification has already been shown to lack 
reproducible results so that different commercial systems will 
generate different subsets of alerts within these categories. 
Empirically estimating the probability of adverse effects in 
different subpopulations provides a more meaningful clinical 
metric that could be used to establish alert thresholds. In this 
study, physicians were more likely to alter treatment when the 
overall risk of injury was high, and, conversely, physicians were 
less likely to respond when the risk was low. To study this 
effect, the threshold to generate an alert was set at a low level of 
risk: an increased risk of 1 per 1000 (0.1%). We identified the 
threshold at which physicians were more likely to respond to an 
alert at an overall risk of 32 per 1000 (3 per 100). In the future, 
empirical estimates of the risk of adverse outcomes could be 
used to establish an alert threshold. Many national agencies 
are establishing methods of conducting robust pharmaco- 
surveillance. Part of the mandate of post-market surveillance 
could be to assess the risk associated with both new drugs and 
drug interactions, thereby providing the data needed to establish 
and calibrate a new generation of smarter drug alert systems. 

An important limitation of drug alerts, including this new 
generation of drug alert systems, is that information is presented 
only on the risk of treatment and not on the potential benefits. 



In the future, efforts should be made to generate empirical 
information on both the risks and benefits of treatment, ideally 
deriving this information from both clinical trials and post- 
market surveillance systems. In addition to providing clinicians 
and patients with relative risk and benefit information, ideally 
individualized to a patient's risk profile, alternate treatment 
options that have lower risk with equivalent benefit for the 
same treatment indication could be provided. Future research 
should be directed to methods of generating empirical estimates 
of comparative risk and benefit that can be incorporated into 
advanced systems of drug decision support, cognitive decision- 
making studies that will enable the best methods of presenting 
this information to physicians and patients to be identified, 
methods of establishing the threshold for generating drug alerts, 
and the potential benefits of displaying alternate therapy 
options by treatment indication. 

In summary, this study shows that individual risk estimates 
displayed graphically and numerically are a more effective 
method of eliciting response to drug alerts and reducing the risk 
of medication-related injury in higher risk seniors. Future 
research should estimate the reduction in injury rate related to 
this new generation of alerts and determine whether similar 
effects can be achieved in other therapy classes for other 
important outcomes. 
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